6 research outputs found
Shared Roots: Regularizing Deep Neural Networks through Multitask Learning
In this paper, we propose to regularize deep neural nets with a new type of multitask learning where the auxiliary task is formed by agglomerating classes into super-classes. As such, it is possible to jointly train the network on the class-based classification problem AND super-class based classification problem. We study this in settings where the training set is small and show that , concurrently with a regularization scheme of randomly reinitializing weights in deeper layers, this leads to competitive results on the ImageNet and Caltech-256 datasets and state-of-the-art results on CIFAR-100
ERM++: An Improved Baseline for Domain Generalization
Multi-source Domain Generalization (DG) measures a classifier's ability to
generalize to new distributions of data it was not trained on, given several
training domains. While several multi-source DG methods have been proposed,
they incur additional complexity during training by using domain labels. Recent
work has shown that a well-tuned Empirical Risk Minimization (ERM) training
procedure, that is simply minimizing the empirical risk on the source domains,
can outperform most existing DG methods. We identify several key candidate
techniques to further improve ERM performance, such as better utilization of
training data, model parameter selection, and weight-space regularization. We
call the resulting method ERM++, and show it significantly improves the
performance of DG on five multi-source datasets by over 5% compared to standard
ERM, and beats state-of-the-art despite being less computationally expensive.
Additionally, we demonstrate the efficacy of ERM++ on the WILDS-FMOW dataset, a
challenging DG benchmark. We hope that ERM++ becomes a strong baseline for
future DG research. Code is released at
https://github.com/piotr-teterwak/erm_plusplus.Comment: An improved baseline for Domain Generalizatio
VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting
Label-efficient and reliable semantic segmentation is essential for many
real-life applications, especially for industrial settings with high visual
diversity, such as waste sorting. In industrial waste sorting, one of the
biggest challenges is the extreme diversity of the input stream depending on
factors like the location of the sorting facility, the equipment available in
the facility, and the time of year, all of which significantly impact the
composition and visual appearance of the waste stream. These changes in the
data are called ``visual domains'', and label-efficient adaptation of models to
such domains is needed for successful semantic segmentation of industrial
waste. To test the abilities of computer vision models on this task, we present
the VisDA 2022 Challenge on Domain Adaptation for Industrial Waste Sorting. Our
challenge incorporates a fully-annotated waste sorting dataset, ZeroWaste,
collected from two real material recovery facilities in different locations and
seasons, as well as a novel procedurally generated synthetic waste sorting
dataset, SynthWaste. In this competition, we aim to answer two questions: 1)
can we leverage domain adaptation techniques to minimize the domain gap? and 2)
can synthetic data augmentation improve performance on this task and help adapt
to changing data distributions? The results of the competition show that
industrial waste detection poses a real domain adaptation problem, that domain
generalization techniques such as augmentations, ensembling, etc., improve the
overall performance on the unlabeled target domain examples, and that
leveraging synthetic data effectively remains an open problem. See
https://ai.bu.edu/visda-2022/Comment: Proceedings of Machine Learning Researc